Claims 



[cl ] 1 . An interactive document processing system, comprising: 

a memory for recording image data of a rendered hardcopy document; 
a display screen; 

a processing device coupled to the display screen; and 
an input device for recording signal data; the input device communicating the 
signal data to the processing device; the signal data representing at least a 
portion of the hardcopy document; 

wherein the processing device in response to receiving the signal data from the 
input device: 

^2 identifying the image data in the signal data using the memory; 

W identifying annotations in the signal data by comparing the image data with the 

|.j signal data; 

3 S [: ; 

\Z modifying the image data recorded in the memory in accordance with the 

Cj annotations. 

; 7 [c2] 2. A system according to claim 1 , further wherein modifying the image data 

pj further comprises: 

modifying the image data in accordance with the command data; 
- = displaying the modified image data on the display screen; and 

responsive to an acceptance signal from the user of the interactive document 

processing system, updating the image data of the hardcopy document in the 

memory with the modified image data. 

[c3] 3. A system according to claim 2, wherein the input device is a camera. 

[c4] 4. A system according to claim 1 , further comprising a stylus. 

[c5] 5. A system according to claim 4, wherein the stylus further comprises: 

an embedded camera; and 

a transmitter that communicates signals from the camera to the processing 
device. 

[c6] 6. A system according to claim 4, wherein the stylus includes an accelerometer 

for recording gestures. 
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[c7] 7. A system according to claim 4, wherein the annotations correspond to 

command data specifying one of editing, navigating, creating cross-references, 
copying, distributing, and filing. 

[c8] 8. A system according to claim 4, wherein the stylus includes a marker for 

laying down marks on the document; and wherein the camera detects the marks 
in order to determine annotations. 

[c9] 9. A system according to claim 4, wherein the stylus includes a pointer for 

simulating marks on the document; and wherein the camera detects the 
simulated marks in order to determine instruction marks. 

j [d 0] 10. A system according to claim 1 , wherein the annotations are defined by 

^ recording one of: markings on the hardcopy document; gestures on the 

i hardcopy document; and signals from a command sheet. 

J tell] 1 1 . A system according to claim 1 , wherein the input device comprises an 

electronic pen with an embedded processing device for performing OCR and 
■£ transmitting textual content to the processing device coupled to the display 

^ screen. 

i t cl 21 1 2. A system according to claim 1 , wherein the input device is adapted to 

record the document and provide position information of the input device 
relative to the hardcopy document. 

[cl 3] 1 3. A system according to claim 1 , further comprising at least one further input 

device for recording user input. 

[cl 4] 14. A system according to claim 1 , further comprising using at least one 

alignment point to identify relationships between at least one of: characters, 
text strings, headings, paragraphs, graphics, shapes of paragraphs, and 
patterns of word spacing; the at least one alignment point providing alignment 
between the signal data and the image data. 

^ 1 5. A system according to claim 1 , wherein the memory records an identifier 

associated with the image data; the identifier being rendered on the hardcopy 
document; the identifier and the image data being recorded in the memory in 
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response to rendering a hardcopy representation of the image data. 

C cl 6 1 1 6. A system according to claim 1 5, wherein the preprocessing device recovers 

the identifier from the signal data and uses the identifier to retrieve the image 
data from the memory. 

t c1 7] 1 7. A system according to claim 1 6, wherein the identifier is one of a bar code, 

a glyph tag, and a paper edge marking. 

[cl 8] 1 8. A method for operating an interactive document processing system having a 

processing device coupled to a display screen, an input device, and a memory, 
comprising: 

(a) recording image data of a rendered hardcopy document in the memory; 

(b) recording signal data with the input device; the signal data representing at 
least a portion of the hardcopy document; 

(c) identifying the image data in the signal data using the memory; 

(d) identifying annotations in the signal data by comparing the image data with 
the signal data; 

(e) modifying the image data recorded in the memory in accordance with the 
annotations. 

[cl 9] 1 9. The method according to claim 1 8, further comprising: 

repeating (b) - (e) for a rendered hardcopy document of the modified image 
data recorded in the memory; and 

recording changes made to the image data at (e) in the modified image data 
recorded in the memory. 



[c20] 



20. A document processing system, comprising: 
a memory; and 

a processor coupled to the memory for receiving a print request; the processor 
satisfying the print request by: 

(a) rendering a hardcopy document of the print request; the hardcopy document 
being rendered with an identifier thereon; 

(b) storing in the memory the identifier and an electronic representation of the 
rendered hardcopy document; 
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(c) providing the electronic representation of the rendered hardcopy document 
in response to receiving a request therefor with the identifier; and 

(d) using the electronic representation to identify annotations applied to the 
rendered hardcopy document. 
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